Re-calibrating Machine Learning Models Using Confidence Interval Bounds

In this article we propose a novel technique for the re-calibration of Machine Learning (ML) models. This technique is based on the computation of confidence intervals for the probability scores provided by any ML model. Compared to existing and commonly used calibration methods, the proposed approach has two important advantages: first, under weak assumptions it provides theoretical guarantees about calibration;second, this method does not require any further data other than the training set used for ML model development. We illustrate the effectiveness of the proposed approach on a benchmark dataset for COVID-19 diagnosis, by comparing the proposed method against commonly used re-calibration techniques. © 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.

Comparative Assessment of Two Data Visualizations to Communicate Medical Test Results Online

As most countries in the world still struggle to contain the COVID-19 breakout, Data Visualization tools have become increasingly important to support decision-making under uncertain conditions. One of the challenges posed by the pandemic is the early diagnosis of COVID-19: To this end, machine learning models capable of detecting COVID-19 on the basis of hematological values have been developed and validated. This study aims to evaluate the potential of two Data Visualizations to effectively present the output of a COVID-19 diagnostic model to render it online. Specifically, we investigated whether any visualization is better than the other in communicating a COVID-19 test results in an effective and clear manner, both with respect to positivity and to the reliability of the test itself. The findings suggest that designing a visual tool for the general public in this application domain can be extremely challenging for the need to render a wide array of outcomes that can be affected by varying levels of uncertainty.

Machine Learning based on laboratory medicine test results in diagnosis and prognosis for COVID-19 patients: A systematic review

The rapid detection of SARS-CoV-2 infections is essential for both diagnostic and prognostic reasons: However, the current gold standard for COVID-19 diagnosis, that is the rRT-PCR test, is affected by long turnaround time, potential shortage of reagents, high false-negative rates and high costs. Thus, Machine Learning (ML) based methods have recently attracted increasing interest, especially when applied to digital imaging (x-rays and CT scans). In this article, we review the literature on ML-based diagnostic and prognostic methods grounding on hematochemical parameters. In doing so, we address the gap in the existing literature, which has so far neglected the application of ML to laboratory medicine. We surveyed 20 research articles, extracted from the Scopus and PubMed indexes. These studies were characterized by a large heterogeneity, in terms of considered laboratory and clinical parameters, sample size, reference population, employed ML methods and validation procedures. Most studies were found to be affected by reporting and replicability issues: Among the surveyed studies, only three reported complete information regarding the analytic methods (units of measure, analyzing equipment), while nine studies reported no information at all. Furthermore, only six studies reported results on independent external validation. In light of these results, we discuss the importance of a tighter collaboration between data scientists and medicine laboratory professionals, so as to correctly characterize the relevant population, select the most appropriate statistical and analytical methods, ensure reproducibility, enable the correct interpretation of the results, and gain actual usefulness by applying ML methods in clinical practice. © 2021 Biomedia. All rights reserved.

Identification of SARS-CoV-2 positivity using machine learning methods on blood count data: External validation of state-of-the-art models

Introduction: The aim of the paper is to present the results from the process of external validation of a number of machine learning (ML) models that had been previously developed to detect SARS-CoV-2 virus positivity on both symptomatic and asymptomatic patients on the basis of the complete blood count (CBC) test. Methods: Briefly, models were trained using a dataset of 816 COVID-19 positive and 920 negative cases collected at the emergency departments of IRCCS Hospital San Raffaele and IRCCS Istituto Ortopedico Galeazzi. 21 parameters, including the results of the CBC analysis, age [60.9 (0.9) years], gender (57% males) and the presence of COVID-19 related symptoms were used. The validation regarded the evaluation of the error rate (through different metrics, including accuracy, sensitivity, specificity and the area under the curve (AUC)) of the models considered. This external validation was conducted on two well balanced datasets coming from two different hospitals in Northern Italy: Desio hospital and Bergamo Papa Giovanni XXIII hospital. 163 positive and 174 true negative patients from Desio, and 104 positive and 145 true negative from Bergamo were included in the validation. Results: The performance of the predictive models is satisfactory as we can report an average AUC of 95% for both external datasets. Conclusion: ML models have been applied to hematological parameters for a more rapid and cost-effective detection of the COVID-19 disease. We make the point that validated models may be useful in the management and early detection of potential COVID-19 patients.

Prediction of ICU admission for COVID-19 patients: A machine learning approach based on complete blood count data

In this article we discuss the development of prognostic Machine Learning (ML) models for COVID-19 progression: specifically, we address the task of predicting intensive care unit (ICU) admission in the next 5 days. We developed three ML models on the basis of 4995 Complete Blood Count (CBC) tests. We propose three ML models that differ in terms of interpretability: two fully interpretable models and a black-box one. We report an AUC of. 81 and. 83 for the interpretable models (the decision tree and logistic regression, respectively), and an AUC of. 88 for the black-box model (an ensemble). This shows that CBC data and ML methods can be used for cost-effective prediction of ICU admission of COVID-19 patients: in particular, as the CBC can be acquired rapidly through routine blood exams, our models could also be applied in resource-limited settings and to get fast indications at triage and daily rounds. © 2021 IEEE.

Development, evaluation, and validation of machine learning models for covid-19 detection based on complete blood count test from 1,624 patients

The amplification of viral RNA by the reverse transcription polymerase chain reaction (rRT-PCR) test is the current gold standard for the confirmation of an infection from coronavirus (COVID-19), the worldwide pandemic that, in the first eight months post-outbreak, has caused almost 730.000 deaths. However, this test presents known limits, such as a long turnaround time, false-negative rates as high as 15-20%, expensive equipment, and a lack of trained personnel. Thus, there is a need for alternative, faster, cheaper, and more accessible tests. We developed a number of machine learning (ML) classification models (Logistic Regression, Naive Bayes, K-Nearest Neighbors, Random Forest and Support Vector Machines) based on the routine complete blood count (CBC) data, from 1,624 patients (52% COVID-19 positive) on admission to the Emergency Department (ED) at the San Raffaele Hospital (OSR) from February 19, to May 31, 2020. This dataset is composed by 21 features including age, gender, suffering from COVID specific symptoms at triage, COVID-19 positivity, and CBC data. The ML models were trained using a training set comprising 80% of the cases (after model selection and hyperparameter optimization) and were evaluated on a test set comprising the remaining cases (20% of the cases). An external dataset (from 58 patients admitted at the ED of the Istituto Ortopedico Galeazzi of Milan, March 5-May 26, 2020) and an internal dataset (data from 54 patients randomly chosen admitted in OSR in 2018), were used for the validation. All models showed a reported accuracy that is consistently higher than 85%. They achieved good performances in symptomatic patients (with both the sensitivity and specificity at approximately 80%) and performed even better in terms of specificity in asymptomatic patients (100% specificity), although the sensitivity was as low as 50%. Our study demonstrates that ML can be applied to CBC as both an adjunct and alternative method to rRT-PCR for the fast and cost-effective identification of COVID-19-positive patients. This is especially useful in developing countries, or in countries facing an extraordinary increase in contagions, as they can suffer from shortages of rRTPCR reagents and specialized laboratories.

Evidence of significant difference in key COVID-19 biomarkers during the Italian lockdown strategy. A retrospective study on patients admitted to a hospital emergency department in Northern Italy

Background. The Lombardy region, Italy, has been severely affected by COVID-19. During the epidemic peak, in March 2020, patients needing intensive care unit treatments were approximately 10% of those infected. This fraction decreased to approximately 2% in the second part of April, and to 0.4% at the beginning of July. In this region, a lockdown strategy was rigidly enforced since March the 9th to May the 18th. COVID-19 is characterized by several biochemical abnormalities whose discrepancy from normal values was associated to the severity of the disease. The aim of this retrospective study was to compare the hematological and biochemical patterns of patients during and after the establishment of a lockdown strategy to verify whether later patients were experiencing a milder COVID-19 course, as observed by several clinicians of the same Hospital. Material and Methods. Two groups of 84 patients, admitted at the emergency department of the San Raffaele Hospital (Milan, Italy), in March and April respectively, homogeneous for distributions of age, gender, and severity of symptom were selected. The laboratory findings (Complete blood count, glucose, creatinine, C-reactive protein, total bilirubin, urea, electrolytes, enzyme activities, hemogasanalysis and CO-Oxymetry data) of the two controlled groups, were analyzed and compared using the two-sample univariate Kolmogorov-Smirnov test revised for family-wise errors using the Bonferroni correction. Results. White blood cell, platelets, lymphocytes and lactate dehydrogenase showed a statistically significant improvement (i.e. closer or within the normal clinical range) in the April group compared to March. Creatinine, C-reactive protein, Calcium and liver enzymes were also pointing in that direction, although the detected differences were not significant. Discussion. Different distribution of laboratory parameters between the two groups are consistent with an increasingly milder disease phenotype. Since comorbidities were similar in the two controlled groups, we can reasonably presume that the enforcement of a lockdown strategy, as well as the widespread use of respiratory protective devices and social distancing may support our findings.